Telegram Group & Telegram Channel
πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:

using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes
Please open Telegram to view this post
VIEW IN TELEGRAM



tg-me.com/csharp_1001_notes/457
Create:
Last Update:

πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:


using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes

BY C# 1001 notes





Share with your friend now:
tg-me.com/csharp_1001_notes/457

View MORE
Open in Telegram


C 1001 notes Telegram | DID YOU KNOW?

Date: |

Among the actives, Ascendas REIT sank 0.64 percent, while CapitaLand Integrated Commercial Trust plummeted 1.42 percent, City Developments plunged 1.12 percent, Dairy Farm International tumbled 0.86 percent, DBS Group skidded 0.68 percent, Genting Singapore retreated 0.67 percent, Hongkong Land climbed 1.30 percent, Mapletree Commercial Trust lost 0.47 percent, Mapletree Logistics Trust tanked 0.95 percent, Oversea-Chinese Banking Corporation dropped 0.61 percent, SATS rose 0.24 percent, SembCorp Industries shed 0.54 percent, Singapore Airlines surrendered 0.79 percent, Singapore Exchange slid 0.30 percent, Singapore Press Holdings declined 1.03 percent, Singapore Technologies Engineering dipped 0.26 percent, SingTel advanced 0.81 percent, United Overseas Bank fell 0.39 percent, Wilmar International eased 0.24 percent, Yangzijiang Shipbuilding jumped 1.42 percent and Keppel Corp, Thai Beverage, CapitaLand and Comfort DelGro were unchanged.

What is Telegram Possible Future Strategies?

Cryptoassets enthusiasts use this application for their trade activities, and they may make donations for this cause.If somehow Telegram do run out of money to sustain themselves they will probably introduce some features that will not hinder the rudimentary principle of Telegram but provide users with enhanced and enriched experience. This could be similar to features where characters can be customized in a game which directly do not affect the in-game strategies but add to the experience.

C 1001 notes from no


Telegram C# 1001 notes
FROM USA